Predictive Visual Models of Physics for Playing Billiards

Imagine a hypothetical person who has never encountered the game of billiards in their life. When introduced to the game, this person may not be very adept at playing the game, but would be capable of inferring the direction in which the cue ball needs to be hit to displace the target ball to a desired location. How can this person make such an inference without any prior billiards-specific experience? One explanation is that humans are aware of the laws of physics, and a strategy for playing billiards can be inferred simply from an understanding of the dynamics of bouncing objects. However, humans do not appear to consciously solve NewtonâĂŹs equations of motion, but rather seem to have an intuitive understanding of how their actions affect the world. In the specific example of billiards, humans can visually hallucinate the trajectory that the ball would follow when force is applied to the ball, and how the trajectory of ball would change when it hits the side of the billiards table or another ball. We term models that can enable the agents to visually anticipate the future states of the world as visual models of physics. How can a visual model physics be used to plan actions? An agent can use this model to visually hallucinate the future states of the world in response to the action perfumed by the agent. The sequence of such visual hallucinations can be thought of as running an internal simulation of the external world. The agent can run multiple internal simulations of the external world by hallucinating the effects of multiple actions. The agent can then execute the action which in simulation leads to the goal state. The idea of using internal models for planning actions is not new and is well known in the control literature in the form of model predictive control [3]. Though past work has met with some success in using internal models of the agent for planning actions, few prior methods have addressed the question of constructing and using models of the external world for planning actions. Internal models of the agent are useful when an agent has to perform tasks in isolation. However, when performing actions in complex environments, models of both the agent and the external world are required. The external world can be substantially more varied, and therefore harder to model.